aiohttp: Reimplement ChunkedClientResponse to correctly handle chunked responses#1094
aiohttp: Reimplement ChunkedClientResponse to correctly handle chunked responses#1094rspeed wants to merge 1 commit intomicropython:masterfrom
Conversation
701eeb7 to
b123d24
Compare
Correctly combines multiple chunks into a single response body. Signed-off-by: Rob Speed <speed.rob@gmail.com>
b123d24 to
abcc45d
Compare
I don't think that's correct. The current code does support multiple chunks, using the This allows you to read in a chunked response in separate chunks, where the chunk sizes are different. Eg the incoming data could be chunked every 256 bytes, but the application reads in 16 byte blocks: resp = session.get(...)
while True:
data = await resp.read(16)
if not data:
# end of stream
break
process_data(data)IMO that's a useful extension to standard aiohttp because most microcontroller applications can't read in megabytes at once. I think the only issue to solve here is handling |
A chunk-encoded response can (and often will) require multiple reads from the response stream, each of a length determined by a value encoded in its first line.
aiohttp.ChunkedClientResponsecurrently treats the first chunk's size as the size of the full data (chunked encoding is specifically intended for situations where the full size isn't known when the transfer starts) and there is only a single chunk.This PR reimplements
aiohttp.CHunkedClientResponsebased on the HTTP/1.1 specification. Chunked responses with more than one chunk will now work correctly. As a side-effect it also fixes #1093.